Server system

ABSTRACT

A system includes a rack and one or more server systems mounted in the rack. A server system includes a chassis with one or more arrays of devices in the chassis. Each array includes mass storage devices and a server device mounted in the array within the chassis of the server system. The server device occupies no more than an equivalent volume of space in the chassis as one of the mass storage devices. A set of mass storage devices of an array and the server device of the array form a logical node, and a server system may include multiple logical nodes in the same chassis. Each array of devices may be located in a sled that can move into and out of the chassis of the server system.

This application is a continuation of U.S. patent application Ser. No.15/058,103, filed Mar. 1, 2016, which is hereby incorporated byreference herein in its entirety.

BACKGROUND

Organizations such as on-line retailers, Internet service providers,search providers, financial institutions, universities, and othercomputing-intensive organizations often conduct computer operations fromlarge scale computing facilities. Such computing facilities house andaccommodate a large amount of server, network, and computer equipment toprocess, store, and exchange data as needed to carry out anorganization's operations. Typically, a computer room of a computingfacility includes many server racks. Each server rack, in turn, includesmany servers and associated computer equipment.

Computer systems typically include a number of components, suchcomponents include printed circuit boards, mass storage devices, powersupplies, and processors. Some known computer systems include aplurality of large, multiple-processor computers that are configuredinto rack-mounted components, and then are subsequently positionedwithin a rack system. Some known rack systems include 40 suchrack-mounted components and such rack systems will therefore generate asmuch as 10 kilowatts of waste heat. Moreover, some known data centersinclude a plurality of such rack systems.

Some computer systems are rack mounted servers that include a number ofhard disk drives (for example, eight or more hard disk drives) toprovide adequate data storage. Typically, the hard disk drives forservers are of a standard, off-the-shelf type. Standard, off-the-shelfhard disk drives are often a cost effective solution for storage needsbecause such hard disk drives can be obtained at relatively low cost.Nonetheless, in server designs using such standard hard disk drives, thearrangement of the hard disk drives may leave a substantial amount ofwasted space in a server chassis. This wasted space, especially whenmultiplied over many servers in a rack, may result in inadequatecomputing or storage capacity for a system.

Some servers that include a number of hard disk drives use a centralizedcontroller to control accesses to the hard disk drives, such as readsand writes. In such server designs using a centralized controller, afailure of the controller or ancillary equipment associated with thecontroller, may cause all the hard disk drives of the server to beunreachable. Thus reducing the reliability of the server.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a perspective view illustrating a rack comprising multipleserver systems, according to some embodiments.

FIG. 2A is a top view of an interior of a server system, according tosome embodiments.

FIG. 2B is a front view of an interior of a server system, according tosome embodiments.

FIG. 2C is a top view of a portion of a backplane of a server system,according to some embodiments.

FIG. 3 is a perspective view illustrating a rack comprising multipleserver systems, according to some embodiments.

FIG. 4A is a top view of an interior of a server system, according tosome embodiments.

FIG. 4B is a front view of an interior of a server system, according tosome embodiments.

FIG. 4C is a front view of a backplane of a server system, according tosome embodiments.

FIG. 5 illustrates a server device that occupies up to an equivalentvolume of space as a mass storage device, according to some embodiments.

FIG. 6 illustrates installing mass storage devices and a server devicein a server system, according to some embodiments.

FIG. 7 illustrates removing a server device from a server system while aportion of the server system is in operation, according to someembodiments.

While the invention is susceptible to various modifications andalternative forms, specific embodiments thereof are shown by way ofexample in the drawings and will herein be described in detail. Itshould be understood, however, that the drawings and detaileddescription thereto are not intended to limit the invention to theparticular form disclosed, but on the contrary, the intention is tocover all modifications, equivalents and alternatives falling within thespirit and scope of the present invention as defined by the appendedclaims. The headings used herein are for organizational purposes onlyand are not meant to be used to limit the scope of the description orthe claims. As used throughout this application, the word “may” is usedin a permissive sense (i.e., meaning having the potential to), ratherthan the mandatory sense (i.e., meaning must). Similarly, the words“include,” “including,” and “includes” mean including, but not limitedto.

DETAILED DESCRIPTION OF EMBODIMENTS

Various embodiments of computer systems, and systems and methods forperforming computing operations, are disclosed. According to oneembodiment, a system for storing data includes a rack and a serversystem coupled to the rack. The server system includes a chassis and anarray of devices mounted within the chassis of the server system. Thearray of devices includes mass storage devices, such as hard diskdrives, mounted in the array. The mass storage devices are coupled to abackplane of the array. In addition, the array includes a server devicemounted in the array and coupled to the backplane of the array, whereinthe server device occupies up to an equivalent volume of space in thechassis as one of the mass storage devices in the array. The serverdevice is configured to control storage operations for the mass storagedevices in the array and comprises a network port for other systems toaccess data stored on the mass storage devices over a network. In someembodiments, an array may include multiple backplanes communicativelycoupled with each other. For example, a server system may include anarray that includes hard disk drives and a server device that controlsaccess to the hard disk drives. The server device may be mounted in thearray along with the hard disk drives and occupy up to an equivalentvolume of space in the chassis of the server system as one of the harddisk drives. One or more backplanes may communicatively couple the harddisk drives of the array with the server device of the array.

According to one embodiment, a server system includes a chassis and anarray of devices mounted in the chassis, wherein the array includes massstorage devices and a server device. The server device iscommunicatively coupled to the mass storage devices of the array and isconfigured to control storage operations of the mass storage devices inthe array. The server system also includes one or more additional arraysof devices mounted in the chassis. Each of the additional arraysincludes mass storage devices mounted in the additional arrays andrespective server devices mounted in each respective additional array.The respective server devices are communicatively coupled to the massstorage devices of the respective arrays and are configured to controlstorage operations of the mass storage devices of the respective arrays.In some embodiments, each of the arrays and additional arrays may form aseparate logical node within the server system.

According to one embodiment, a method includes coupling mass storagedevices in respective slots of an array of slots in a chassis of aserver system, coupling a server device in a particular slot of thearray of slots in the chassis of the server system, wherein theparticular slot of the array occupies up to an equivalent volume ofspace in the chassis as other ones of the slots of the array in whichthe mass storage devices are coupled, wherein the server device isconfigured to control storage operations for the mass storage devices inthe array of slots and comprises a network port for communicating withother systems.

As used herein, “backplane” means a plate or board to which otherelectronic components, such as mass storage devices, server devices,etc. can be mounted. In some embodiments, mass storage devices, whichcan include one or more hard disk drives, are plugged into a backplanein a generally perpendicular orientation relative to the face of thebackplane. In some embodiments, a server device is plugged into abackplane in a generally perpendicular orientation relative to the faceof the backplane. In some embodiments, a backplane includes one or morepower buses that can transmit power to components on the backplane, andone or more data buses that can transmit data to and from componentsinstalled on the backplane.

As used herein, a “cable” includes any cable, conduit, or line thatcarries one or more conductors and that is flexible over at least aportion of its length. A cable may include a connector portion, such asa plug, at one or more of its ends.

As used herein, “circuit board” means any board or plate that has one ormore electrical conductors transmitting power, data, or signals fromcomponents on or coupled to the circuit board to other components on theboard or to external components. In certain embodiments, a circuit boardis an epoxy glass board with one or more conductive layers therein. Acircuit board may, however, be made of any suitable combination ofmaterials.

As used herein, “chassis” means a structure or element that supportsanother element or to which other elements can be mounted. A chassis mayhave any shape or construction, including a frame, a sheet, a plate, abox, a channel, or a combination thereof. In one embodiment, a chassisis made from one or more sheet metal parts. A chassis for a computersystem may support circuit board assemblies, power supply units, datastorage devices, fans, cables, and other components of the computersystem.

As used herein, “computing” includes any operations that can beperformed by a computer, such as computation, data storage, dataretrieval, or communications.

As used herein, “data center” includes any facility or portion of afacility in which computer operations are carried out. A data center mayinclude servers dedicated to specific functions or serving multiplefunctions. Examples of computer operations include informationprocessing, communications, testing, simulations, power distribution andcontrol, and operational control.

As used herein, “mounting” a particular element on another elementrefers to positioning the particular element to be in physical contactwith the other element, such that the other element provides one or moreof structural support, positioning, structural load transfer,stabilization, shock absorption, some combination thereof, or the likewith regard to the particular element. The mounted particular elementmay be positioned to rest upon one or more upper surfaces of the otherelement, independent of coupling the elements via one or more couplingelements. In some embodiments, mounting the particular element toanother element includes coupling the elements such that the otherelement provides one or more of structural support, positioning,structural load transfer, stabilization, shock absorption, somecombination thereof, or the like with regard to the particular element.

As used herein, a “rack” means a rack, container, frame, or otherelement or combination of elements that can contain or physicallysupport one or more computer systems. In some embodiments a rack is astandard 19″ rack that conforms to EIA standards.

Some servers that mount in a rack may be general purpose servers, whilesome rack-mounted servers may be specially designed for storagecapacity. Such specially designed servers may include storage serversthat include several hard disk drives and controller servers thatinclude controllers that manage storage operations directed at the harddisk drives in the storage servers. A controller server may be mountedin a separate chassis in a separate rack slot from a rack slot in whicha storage server is mounted. In such servers, some available space inthe slot of the rack in which the controller server is mounted may gounutilized, thus reducing the number and density of hard disk drivesthat can be mounted in the rack. For example, only a portion of a spaceof a rack slot in which a controller server is mounted may be occupiedby components of the controller server or only a portion of an interiorspace within a chassis of a controller server may be occupied bycomponents of the controller server. Also, some servers that include aseparate controller server may be configured such that the controllerserver and hard disk drives controlled by the controller server form asingle logical node. In such servers, if a common component fails, suchas a component in the controller server, the whole logical nodeincluding all of the hard disk drives controlled by the controllerserver may become inaccessible. Thus in such designs a large number ofhard disk drives may be rendered unavailable due to a single componentfailure.

In some embodiments, wasted space in a separate controller server may beeliminated by including a server device in a same chassis of a serversystem along with mass storage devices, such as hard disk drives, thatare controlled by the server device. Also, failure impact due to asingle component failure may be reduced by including several logicalnodes, each with their own server device, in the same chassis of theserver system. In some embodiments, a server system may include a serverdevice mounted in a same chassis with mass storage devices, such as harddisk drives. The server device may occupy an equivalent volume of spacein the chassis as one of the mass storage devices. Thus a greaterdensity of mass storage devices per rack may be achieved by eliminatingunutilized space of a separate controller server as described above.Also, in some embodiments, multiple logical nodes each comprising itsown server device and an array of mass storage device controlled by theserver device may be included in a chassis of a server system. Themultiple logical nodes may be independent of each other, so that if acomponent in one logical node fails, such as a server device of thelogical node, the other logical nodes in the server system may not beaffected by the failure and continue to be available to perform storageoperations.

A chassis of a server system may be a single enclosure that isconfigured to mount in a rack. For example, mass storage devices andserver devices mounted in a chassis of a server system may be installedin a rack as a single whole system or device that occupies a slot in therack. In some embodiments, a height of a server system may span multiple1 Us in a standard 19″ EAI rack. For example, a server system may have aheight of 1 U, 2 U, 3 U, 4 U, 5 U, 1.5 U, 2.5 U, etc.

FIG. 1 is a perspective view illustrating a rack comprising multipleserver systems, according to some embodiments. System 100 includes rack102 and server systems 104, 106, and 108 mounted in rack 102. Each ofserver systems 104, 106, and 108 include multiple arrays of device slotswith mass storage devices and a server device mounted in respectivedevice slot of respective ones of the arrays. Each array of mass storagedevices and a respective server device forms a separate logical node ofmultiple logical nodes of the server systems. For example, server system104 includes 9 logical nodes 110, wherein each logical node includes anarray of mass storage devices and a server device for controllingstorage operations directed at the array of mass storage devices. Insome embodiments, each logical node of a server system includes massstorage devices and a server device mounted in device slots of a sled,wherein the sled is configured to move into and out of a chassis of theserver system independently of other components of the server system,such as other sleds comprising other logical nodes. FIG. 1 illustratessled 112 partially extended out of chassis 114 of server system 104.Mass storage devices 116 and server device 118 are mounted in deviceslots of an array of device slots of sled 112. Each of logical nodes 110may include a separate array of device slots with mass storage devicesand a server device mounted in the array of device slots as shown insled 112. In some embodiments, a server device, such as server device118, occupies up to an equivalent volume of space within chassis 114 asone of the mass storage devices of the server system, such as one ofmass storage devices 116. In some embodiments, a server device, such asserver device 118, may have a form factor that is equivalent to a formfactor of a mass storage device, such as one of mass storage devices116. In some embodiments, a server device, such as server device 118,may have a form factor that is smaller than a form factor of a massstorage device, such as one of mass storage devices 116. For example,server device 118 may have a form factor such that server device 118 canfit within a space that has a volume equivalent to a volume of a massstorage device, such as one of mass storage devices 116.

A server device, such as server device 118, may include one or morenetwork ports and be configured to communicate with systems outside ofserver system 100. For example, server device 118 may be configured toreceive read and write requests from other computing systems and/orservers to read data from and write data to mass storage devices 116.

In some embodiments, an array of devices in a server system that formsits own logical node, such as one of logical nodes 110 in server system104, may include more than one server device in the logical node. Forexample, a logical node, such as one of logical nodes 110, may includemultiple server devices in the same array of devices that forms thelogical node. The multiple server devices of a logical node may beconfigured to coordinate with each other to control storage operationsrelated to mass storage devices of the logical node, such as massstorage devices 116. In some embodiments, a ratio of server devices tomass storage devices may be adjusted by adding or removing mass storagedevices and/or server devices from an array of devices that forms thelogical node.

In some embodiments, server devices of a logical node may performcompute operations in addition to managing storage operations related toan array of mass storage devices of the logical node. In someembodiments, multiple server devices may be included in a single logicalnode to provide additional computing capacity for the logical node.

In some embodiments, mass storage devices and/or a server device of alogical node are configured to be removed from the logical node whileother logical nodes of the server system are in operation. For example,a mass storage device has been removed from a device slot of sled 112.In some embodiments, mass storage devices of a logical node may be hotpluggable, such that a mass storage device can be removed from a logicalnode while other mass storage devices of the logical node remainavailable to perform storage operations and the server device of thelogical node remains in operation. For example, mass storage devices 116and server device 118 of sled 112 may perform storage operations whilesled 112 is partially removed from server system 104 and while a massstorage device is removed from one of the device slots of sled 112.Also, in some embodiments, a server device, such as server device 118may be removed from a device slot of a server system, while otherlogical nodes, such as other ones of logical nodes 110 remain inoperation. For example, server device 118 may be removed from a deviceslot of sled 112 while other ones of logical nodes 110 of server system104 remain in operation.

In some embodiments, data storage operations for mass storage devices ofa logical node, such as one of logical nodes 110, may be controlled by aserver device included in the logical node in an array of devices slotsof the logical node along with a mass storage devices of the logicalnode mounted in the array of device slots without using an additionalcontroller. The mass storage devices of a logical node may be controlledwithout an external controller controlling storage operations of themass storage devices of the logical node. For example each of logicalnodes 110 may include their own server device and not rely on anexternal controller mounted in another rack slot in rack 102 to controldata operations of mass storage devices in respective ones of logicalnodes 110.

By including server devices in an array with mass storage devices in asingle chassis, a high density of mass storage devices in a rack can beachieved. For example, in some embodiments, a 48 U standard 19″ rack mayinclude over 1,440 3.5″ hard disk drives along with server devices forcontrolling the hard disk drives, wherein the hard disk drives aremounted in device slots of multiple arrays included in multiple serversystems mounted in the rack and wherein each array includes a serverdevice mounted in a device slot of the respective array. In someembodiments, a 48 U standard 19″ rack may include more or less hard diskdrives mounted in device slots of multiple arrays. In some embodiments,other rack sizes may be used.

In some embodiments, a server system may include arrays of device slotsthat run from a front side of the server system to a back side of theserver system. For example, FIGS. 2A-2C illustrate an embodiment of aserver system with multiple arrays of device slots with mass storagedevices and respective server devices mounted in the respective arrays,wherein the arrays run from a front side of the server system to a backside of the server system.

FIG. 2A is a top view of an interior of a server system, according tosome embodiments. Server system 200 includes chassis 202, mass storagedevices 204 and server devices 205. Mass storage devices 204 and serverdevices 205 are mounted in device slots of arrays 206, 208, 210, 212,214, 216, 218, 220, and 222. Each of arrays 206, 208, 210, 212, 214,216, 218, 220, and 222 includes its own server device 205 and forms aseparate logical node of server system 200. In some embodiments, each ofarrays 206, 208, 210, 212, 214, 216, 218, 220, and 222 include aseparate sled that is configured to move into and out of chassis 202independently from one another. In some embodiments, a server system,such as server system 200, may include 9 arrays of device slots with 16device slots in each array. In some embodiments, a server system, suchas server system 200 may have a height of 4 U or 5 U. In someembodiments, a server system may have different heights and may includemore or less device slots in a chassis or more or less arrays.

In some embodiments, a backplane may communicatively couple mass storagedevices of a logical node with a server device of the logical node. Insome embodiments, cables may be used to communicatively couple massstorage devices of a logical node with a respective server device of thelogical node without using a backplane. In some embodiments, both abackplane and cables may be used to communicatively couple mass storagedevices of a logical node with a respective server device of the logicalnode.

FIG. 2B is a front view of an interior of a server system, according tosome embodiments. FIG. 2B illustrates backplanes 224 mounted beneathmass storage devices 204 and server devices 205 of arrays 206, 208, 210,212, 214, 216, 218, 220, and 222 in a space between chassis 202 and massstorage devices 204 and server devices 205.

In some embodiments, mass storage devices and a respective server deviceof an array of a server system may be coupled together via a backplane.In some embodiments, a backplane of an array may be mounted beneath massstorage devices and a server device of the array in a space between abottom of the chassis and the bottom of the mass storage devices andserver device. In some embodiments, a backplane may transmit databetween components coupled to the backplane, such as mass storagedevices and a server device. A backplane may include a data connectionthat couples with a server device, such as one of server devices 205. Insome embodiments a server device, such as one of server devices 205 maybe connected to a backplane via a connector that is similar to aconnector used to connect mass storage devices to the backplane and insome embodiments, a separate data connection may be used to couple abackplane with a storage server via a cable. In some embodiments aconnector used to connect mass storage devices to a backplane may beconfigured to accept both mass storage devices and server devices suchthat mass storage device and server devices can be interchangeablycoupled to a backplane via the same connector. In some embodiments aconnector for coupling a mass storage device and/or a connector forcoupling a server device to a backplane may comprise one or more keys orother particular configurations that prevent server devices and massstorage devices from being interchangeably coupled to the sameconnector.

FIG. 2C is a top view of a portion of a backplane of a server system,according to some embodiments. Backplane 224 may be the same backplane224 illustrated in FIG. 2B. Backplane 224 includes connectors 226mounted on circuit board 228. In some embodiments, ones of connectors226 may be configured to only couple with mass storage devices or may beconfigured to only couple with server devices. In other embodiments,ones of connectors 226 may be configured to interchangeably couple withmass storage devices and server devices, such as mass storage devices204 and server devices 205. Backplane 224 also includes power connection230 and data connection 232. Power connection 230 may be configured tocouple to a power source that provides power to server system 200 anddata connection 232 may couple with a server device, such as one ofserver devices 205. A server device, such as server device 205, may inturn be coupled to a larger network such as a rack level network, datacenter network, or other network. For example, data operations fromclients of a server system 200 may be directed at server devices 205 ofserver system 200 and data received and provided by server system 200may be received from or provided to clients of server system 200 viaserver devices 205 and data connections 232. In some embodiments, abackplane may include a data connection configured to couple thebackplane to a larger network such as a rack level network, data centernetwork, or other network. In some embodiments, power may be distributedfrom power connection 230 via circuit board 228 to components coupled tobackplane 224 such as mass storage devices 204 and a server device 205.In some embodiments, a server system, such as server system 200 may notinclude a backplane 224 and may include cables to distribute power tomass storage devices 204 and a server device 205. In some embodiments,cables may transmit data between mass storage devices 204, a serverdevice 205, and clients of the server system. In some embodiments bothcables and a backplane may be used to distribute power and transmit datawithin a server system 200.

In some embodiments, components of a server system, such as serversystem 200, may be cooled by air moving devices such as fans. Forexample in some embodiments, fans may be mounted in space 234 in chassis202 of server system 200 to direct air to flow over mass storage devices204 and server devices 205. In some embodiments, a server system, suchas server system 200, may be liquid cooled. For example, server system200 may include pipes, tubes, or other suitable means to convey a liquidcoolant to mass storage devices 204 and server devices 205 to removewaste heat from mass storage devices 204 and server devices 205. In someembodiments, a server system, such as server system 200, may be cooledby an external cooling system, such as a rack level cooling system. Insome embodiments, other suitable means may be used to remove waste heatfrom mass storage devices and server devices of a server system.

The arrangement of mass storage devices and server devices illustratedin FIGS. 1-2 is an example of a server system with mass storage devicesand server devices mounted in arrays in a common chassis. However, insome embodiments, mass storage devices and server devices may bearranged in other arrangements in a server system. For example, in someembodiments, mass storage devices and respective server devices may bearranged in arrays that span a width of a server system, for examplefrom a left side of the server system to a right side of the serversystem when the server system is mounted in a rack. For example, FIG. 3is a perspective view illustrating a rack comprising multiple serversystems, according to some embodiments. System 300 includes rack 302 andserver systems 304, 306, and 308. Server system 304 includes chassis316, mass storage devices 314, server device 310, and backplane(s) 312.Mass storage devices 314 and server device 310 are mounted in an arrayof device slots in chassis 316 that spans a width of rack 302, e.g. froma left side of rack 302 to a right side of rack 302. Rack 302 isillustrated with ellipses between server system 306 and server system308 to indicate that a rack, such as rack 302, may include more or lessserver systems. In some embodiments, rack 302 may be a 48 U rack andeach server system may have a height that occupies 3 Us of rack 302. Insome embodiments, a server system may have different heights. For easeof illustration, server system 304 is shown partially removed from rack302. In some embodiments, a server system, such as server system 304,may include multiple arrays of mass storage devices and respectiveserver devices that span a width of the server system. For example, insome embodiments, server system 304 may include 6 arrays of serversystems. Also, for ease of illustration, only server system 304 is shownpartially removed from rack 302. However, in some embodiments, serversystems 306 and 308 may include similar components in a similararrangement as shown for server system 304.

FIGS. 4A-C illustrate different views of a server system with arrays ofmass storage devices and server devices arranged in horizontal arrays,according to some embodiments. Any of server systems 304, 306, and 308may include similar components in a similar arrangement as illustratedin FIGS. 4A-C.

FIG. 4A is a top view of an interior of a server system, such as one ofserver systems 304, 306, or 308, according to some embodiments. Serversystem 400 includes mass storage device 402, server devices 404, andbackplanes 406. In server system 400, groups of mass storage devices402, respective ones of server devices 404 and respective backplanes 406are organized in multiple arrays that each form a logical node withinserver system 400. For example server system 400 includes arrays 408,410, 412, 414, 416, and 418. Each of arrays 408, 410, 412, 414, 416, and418 includes a group of mass storage devices 402 and a respective serverdevice 404 that coordinates data storage operations of the mass storagedevices 402 in the respective array. Because each array it its ownlogical node, a failure of a component, such as a server device in onearray does not prevent other arrays or logical nodes within serversystem 400 from continuing to perform storage operations.

In FIG. 4A, a backplane 406 is illustrated as being coupled to each oneof arrays 408, 410, 412, 414, 416, and 418. In some embodiments, asingle back plane may be coupled to multiple arrays. For example, abackplane may be mounted between successive arrays in a chassis of aserver system and mass storage devices and server devices may be coupledto two sides of the backplane. For example a backplane between arrays416 and 418 may be coupled to mass storage devices 402 and server device404 of array 418 on a first side of the backplane and may be coupled tomass storage devices 402 and server device 404 of array 416 on a secondside of the backplane. In some embodiments, mass storage devices andserver devices of respective arrays may be coupled together via cableswithout using a backplane. In some embodiments, both cables andbackplanes may be used to couple together mass storage devices andserver devices. For example, server device 404 may include multipleports, a first port may be coupled via a cable to a first backplane thatis coupled with a portion of mass storage devices 402 in array 418 andserver device 404 may be coupled via another cable to another backplanethat is coupled to another portion of mass storage devices 402 in array418. In some embodiments, other suitable means may be used to couplemass storage devices with a server device that coordinates storageoperations associated with the mass storage devices.

FIG. 4B is a front view of an interior of a server system, according tosome embodiments. FIG. 4B illustrates a front view of an interior ofserver system 400 and backplane 406 coupled to mass storage devices 402and server device 404. As shown in FIG. 4B, in some embodiments multiplebackplanes 406 may be coupled to mass storage devices and a serverdevice. FIG. 4C is a front view of a backplane of a server system,according to some embodiments. FIG. 4C illustrates one of backplanes406. A backplane may include multiple connectors, such as connectors422. A backplane may also include power connectors, such as powerconnector 424, and data connectors, such as data connectors 426. In someembodiments, a server system may be connected to a larger network via aserver device, such as one of server devices 404. The server device mayin turn be connected to mass storage devices, such as mass storagedevices 402 via cables connected to data connectors 426. In someembodiments, connectors of a backplane, such as connectors 422 maycouple to mass storage devices and one of the connectors may couple witha server device, such as server device 404. In some embodiments, aserver device, such as server device 404, may be coupled to a backplane,such as backplane 406 via a cable connector, such as one of dataconnectors 426. Another data connector of a backplane, such as anotherone of data connectors 426, may couple the backplane with an additionalbackplane.

In some embodiments, components of a server system, such as serversystem 400, may be cooled by air moving devices such as fans. Forexample in some embodiments, fans may be mounted in or couple withchassis 420 of server system 400 to direct air to flow over mass storagedevices 402 and server devices 404. In some embodiments, a serversystem, such as server system 400, may be liquid cooled. For example,server system 400 may include pipes, tubes, or other suitable means toconvey a liquid coolant to mass storage devices 402 and server devices404 to remove waste heat from mass storage devices 402 and serverdevices 404. In some embodiments, a server system, such as server system400, may be cooled by an external cooling system, such as a rack levelcooling system. In some embodiments, other suitable means may be used toremove waste heat from mass storage devices and server devices of aserver system.

FIG. 5 illustrates a server device that occupies an equivalent volume ofspace as a mass storage device, according to some embodiments. Serverdevice 500 illustrated in FIG. 5 may be any of the server devicesdescribed in FIGS. 1-4. In some embodiments, server device 500 may havea form factor that occupies up to an equivalent volume of space as amass storage device, such as a 3.5″ hard disk drive.

Server device 500 includes system on a chip 502, memory devices 510,power connector 508, network interface controller 504, network port 506,management and control circuit 512, boot drive 514, and connectors 516and 518. System on a chip 502 comprises a processor and interface forcommunicating with mass storage devices via connectors 516 and 518.System on a chip 502 also comprises a communication interface withmemory 510. In some embodiments system on a chip 502 may be an SoC, ageneral purpose processor, an application specific integrated circuit(ASIC), or another device that provides processing capability andcommunicates with connectors 516 and 518. In some embodiments,management and control circuit 512 manages conditions of server device500 such as temperature conditions. In some embodiments, management andcontrol circuit 512 includes an interface for connecting with anexternal heath monitoring system, such as an intelligent platformmanagement interface (IPMI). In some embodiments, memory devices 510 areRAM devices such as double data rate fourth generation synchronousdynamic random-access memory (DDR4) or other suitable RAM devices. Insome embodiments, network port 506 is a small form factor pluggable 10gigabit port or other suitable port for network connection. In someembodiments, power connector 508 may be configured to receive electricalpower via a cable connection, and in some embodiments, power connector508 may be configured to couple with a backplane and receive electricalpower via the backplane. In some embodiments, boot drive 514 is a solidstate drive that stores program instructions for booting server device500. In some embodiments, boot instructions for server device 500 may bestored in a remote location and boot drive 514 may be omitted. In someembodiments connectors 516 and 518 are mini SAS HD connectors. In someembodiments connectors 516 may include two mini SAS HD connectors withfour lanes each and connectors 518 may include two mini SAS HDconnectors with four lanes each. Thus connectors 516 and 518 maycollectively include 16 lanes for connecting with 16 mass storagedevices. In some embodiments, other suitable connectors may be used forconnectors 516 and 518. In some embodiments, communication viaconnectors 516 and 518 may be in accordance with a FibreChannelstandard, Serial ATA standard, or other suitable standard. In someembodiments, a backplane may couple with connectors 516 and 518 todirectly couple server device 500 with mass storage devices via abackplane without using expanders or host bus adapters to connect serverdevice 500 to mass storage devices.

In some embodiments, a server device, such as server device 500 may bemounted in a device slot of an array of device slots with connectors 516and 518 oriented down towards a bottom of the device slot and coupledwith a backplane that runs beneath mass storage devices of the array andthe server device mounted in the device slot. In some embodiments, aserver device, such as server device 500 may be mounted in a device slotwith connectors 516 and 518 oriented parallel to the device slot. Abackplane may be positioned between successive arrays of mass storagedevices and server devices so that connectors 516 and 518 couple withthe backplane that is positioned between successive arrays of massstorage devices and server devices. In some embodiments, a serverdevice, such as server device 500 may be coupled to backplanes viacables connected to connectors 516 and 518.

In some embodiments, a server device such as server device 500 may beintegrated into a backplane. For example the respective components ofserver device 500 described above may be mounted on a circuit board of abackplane. In some embodiments, a server device, such as server device500 may be liquid cooled.

In some embodiments, a server device, such as server device 500, mayhave a form factor that is equivalent to a form factor of a standard3.5″ hard disk drive. In some embodiments a server device, such asserver device 500, may have a different form factor.

FIG. 6 illustrates installing mass storage devices and a server devicein a server system, according to some embodiments. At 602 mass storagedevices are coupled in respective device slots of an array of deviceslots within a chassis of a server system. In some embodiments, a serversystem may include multiple arrays of device slots within the chassis ofthe server system. In some embodiments, device slots may be included ina sled of a server system and coupling mass storage devices inrespective device slots of a server system may include coupling the massstorage devices into device slots of the sled of the server system.

At 604 a server device is coupled in a device slot of the array ofdevice slots. In some embodiments a server system chassis may includemultiple arrays of devices slots and a separate server device may becoupled in a device slot of each array. In some embodiments each arraythat includes mass storage devices and a respective server device mayform its own logical node within the server system.

At 606, the mass storage devices of the array are communicativelycoupled with the server device of the array. In some embodiments eacharray may include its own server device communicatively coupled withmass storage devices of the array. In some embodiments a server deviceand mass storage devices may be communicatively coupled via a backplane.In some embodiments, cables may communicatively couple a server devicewith mass storage devices of an array. In some embodiments, multiplebackplanes and cables may be used to communicatively couple a serverdevice with mass storage devices. In some embodiments, an array mayinclude 16 device slots, with one of the device slots occupied by aserver device and the remaining 15 device slots occupied by mass storagedevices. In some embodiments, a server system may include 6, 9, or someother number of arrays of device slots in a common chassis. In someembodiments, mass storage devices of the array and a server device ofthe array may be communicatively coupled when the mass storage devicesand the data server device are coupled in respective slots in the array.For example, 606 may be performed as part of performing 602 and 604.

As discussed at 604 and 606, in some embodiments, a server system mayinclude multiple arrays of device slots. At 608 additional mass storagedevices and additional server devices are coupled in device slots ofadditional arrays of device slots within a server system.

At 610, the server system is coupled in one or more slots of a rack. Insome embodiments, a server system may occupy 3 U, 4 U, or 5 U of rackspace in a rack. In some embodiments a server system may have differentheights. In some embodiments, multiple server systems may be mounted ina rack.

FIG. 7 illustrates removing a server device from a server system while aportion of the server system is in operation, according to someembodiments. At 702, a sled of a server system is at least partiallyremoved from a chassis of the server system. In some embodiments, aserver system may include multiple logical nodes in a common chassis.Each sled of a server system may comprise its own logical node, so thatpartially removing a sled from a server system does not affect storageoperations of other logical nodes mounted in other sleds within thechassis of the server system. In some embodiments, other logical nodesmay continue to perform storage operations while a particular sledcomprising a particular logical node of mass storage devices and anassociated server device are at least partially removed from a chassisof a server system.

At 704, a server device is removed from the sled that has been partiallyslid out of the chassis of the server system. In some circumstances, amass storage device may be removed from the sled instead of the serverdevice. In some embodiments, mass storage devices of a server system maybe hot pluggable such that a mass storage device can be removed from andadded to an array of device slots while remaining mass storage devicescommunicatively coupled with a server device associated with the array(or sled) may continue to perform data storage operations.

At 706, a different server device or the same server device is installedin the device slot from which the server device was removed. In somecircumstances, it may be desirable to replace or repair a server device,for example due to a failure associated with the server device. In someembodiments, an additional server device may be installed in anotherslot of the sled. For example, additional compute capacity may bedesired for the logical node that corresponds with the particular sledthat is partially removed. In some embodiments, device slots of a sledmay be configured to accept both mass storage devices and serverdevices, such that a mass storage device can be removed from a sled,such as the sled that is partially removed, and replaced with a serverdevice to increase a computing capacity of a logical node correspondingto the sled.

At 708, the sled is re-inserted into the chassis of the server system.At 710 data storage operations are performed in relation to the massstorage devices mounted in the device slots of the sled using the new orreturned server device.

In some embodiments, mass storage devices in a server system arestandard, off-the-shelf hard disk drives. Examples of suitable hard diskdrive form factors may include 3.5″, 5.25″, and 2.5″. In one embodiment,a standard 3.5″ hard disk drive is installed in a device slot of aplurality of device slots of an array along with other standard 3.5″hard disk drives. In some embodiments, a server device of an array mayhave a form factor equivalent to a 2.5″, 3″, 3.5″, or 5.25″ hard diskdrive.

Although in the embodiments described above, some of the server systemshave been described as being 3 U, 4 U, or 5 U in height, server systemsmay in various embodiments be 2 U, 4 U, 5 U, 6 U or any other height ordimensions.

The various methods as illustrated in the figures and described hereinrepresent example embodiments of methods. The order of method may bechanged, and various elements may be added, reordered, combined,omitted, modified, etc.

Although the embodiments above have been described in considerabledetail, numerous variations and modifications will become apparent tothose skilled in the art once the above disclosure is fully appreciated.It is intended that the following claims be interpreted to embrace allsuch variations and modifications.

1.-20. (canceled)
 21. A system, comprising: a rack; a server systemcoupled to the rack wherein the server system comprises: a chassisconfigured to couple in a slot of the rack; a plurality of arrays ofdevices mounted in the chassis, wherein the respective arrays of devicesform respective independent server nodes of the server system, andwherein each array of devices comprises: a plurality of mass storagedevices mounted in the array; and one or more server devices mounted inthe array with the mass storage devices of the array; wherein therespective server devices of the respective server nodes are configuredto control storage operations for the mass storage devices of therespective service nodes independent of the other server devices ofother ones of the server nodes such that a failure of a given serverdevice of a given server node does not limit operation of other ones ofthe server nodes.
 22. The system of claim 21, wherein a height of theserver system is such that the server system is coupled in a 1 U slot ofthe rack.
 23. The system of claim 21, further comprising: a plurality ofsleds, wherein for each array of devices, the mass storage devices ofthe array and the server device of the array are mounted in one of thesleds.
 24. The system of claim 23, wherein the respective arrays mountedin the respective sleds are configured to be independently moved intoand out of the chassis of the server system independently of other onesof the arrays mounted in other ones of the sleds.
 25. The system ofclaim 21, wherein the mass storage devices and the server devices of therespective arrays that form respective independent server nodes areconfigured to be hot pluggable such that a mass storage device or aserver device of a given array can be removed from or added to the givenarray while other mass storage devices and server devices of the givenarray continue to perform data storage operations of the server nodecorresponding to the given array.
 26. The system of claim 21, wherein afirst one of the arrays that forms a first one of the server nodescomprises a ratio of mass storage devices to server devices that isdifferent from a ratio of mass storage devices to server devices foranother one of the arrays that forms another one of the server nodes.27. The system of claim 26, further comprising: connectors for therespective arrays of devices, wherein the connectors of the respectivearrays are configured to interchangeably couple with a server device ora mass storage device, such that a ratio of mass storage devices toserver devices of a given array is adjustable by replacing a serverdevice in the array with a mass storage device or by replacing a massstorage device with a server device.
 28. The system of claim 27, whereinthe server device of the given array occupies up to an equivalent volumeof space in the chassis as one of the mass storage devices in the givenarray.
 29. A multi-node storage server, comprising: a chassis configuredto couple in a rack slot of a rack; a plurality of arrays of devicesmounted in the chassis, wherein the respective arrays of devices formrespective independent server nodes of the server system, and whereineach array of devices comprises: a plurality of mass storage devicesmounted in the array; and one or more server devices mounted in thearray with the mass storage devices of the array; wherein the respectiveserver devices of the respective server nodes are configured to controlstorage operations for the mass storage devices of the respectiveservice nodes independent of the other server devices of other ones ofthe server nodes such that a failure of a given server device of a givenserver node does not limit operation of other ones of the server nodes.30. The multi-node storage server of claim 29, wherein respective onesof the server devices of the arrays of devices occupy up to anequivalent volume of space in the chassis as one of the mass storagedevices of the arrays of devices.
 31. The multi-node storage system ofclaim 29, comprising: connectors for the respective arrays of devices,wherein the connectors are configured to interchangeably couple with aserver device or a mass storage device, such that a ratio of massstorage devices to server devices of a given array is adjustable byreplacing a server device in the array with a mass storage device or byreplacing a mass storage device of the given array with a server device.32. The multi-node storage system of claim 29, wherein one of the arraysof devices of the multi-node storage system comprises a greater numberof server devices than another one of the arrays of devices of themulti-node storage system.
 33. The multi-node storage system of claim29, wherein respective ones of the arrays of devices comprise cablesthat couple the mass storage devices of the array to the server deviceof the array.
 34. The multi-node storage system of claim 29, whereinrespective ones of the arrays of devices comprise respective back planesconfigured to couple the mass storage devices of the array to the serverdevice of the array.
 35. The multi-node storage system of claim 34,further comprising: one or more double-sided backplanes, wherein for twoadjacent arrays of devices mounted in the chassis, a double-sidedbackplane is configured to: couple together mass storage devices and aserver device of a first array on a first side of the double-sidedbackplane; and couple together mass storage devices and a server deviceof a second array on a second side of the double-sided backplane.
 36. Amethod, comprising: providing a multi-node storage system comprising: achassis configured to couple in a rack slot of a rack; a plurality ofarrays of devices mounted in the chassis, wherein the respective arraysform respective independent server nodes of the server system, andwherein each array comprises: a plurality of mass storage devicesmounted in the array of devices; and one or more server devices mountedin the array with the mass storage devices of the array of devices; andindependently controlling storage operations for the mass storagedevices of the respective arrays, via respective server devices of therespective arrays, such that a failure of a given server device of agiven array does not limit operation of mass storage devices of otherones of the arrays.
 37. The method of claim 36, comprising: at leastpartially removing one of the arrays of devices from the chassis whileother ones of the arrays of devices remain in operation; and removing aserver device or mass storage device from the one array of devices. 38.The method of claim 37, further comprising: replacing the removed serverdevice or mass storage device with a replacement server device or massstorage device, while the other ones of the arrays of devices remain inoperation.
 39. The method of claim 37, wherein replacing the removedserver device or mass storage device changes a ratio of server devicesto mass storage devices for the one of the arrays of devices.
 40. Themethod of claim 39, wherein the array of devices comprises connectorsconfigured to interchangeably couple with a mass storage device or aserver device to couple the mass storage device or the server device inthe array of devices.